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Abstract 
This paper on a quasi-experimental pilot study of the impacts of a high school teacher 
professional development (PD) series. The 10-session series, “Engaging High School Students in 
Academic Work,” was designed to equip teachers to deepen students’ learning and engagement 
and thereby increase course-passing rates. The study took place in a district in the US. 
Southwest that selected two of its interested high schools with similar demographic 
characteristics to participate, one receiving the PD and the other serving as the comparison 
school. Longitudinal analyses found statistically and educationally significant impacts of the PD 
series on four of the five measures in the primary outcome domain (students’ course passing) and 
on teachers’ reports of providing extra help. The article discusses limitations of the study and the 
need for further exploration of how PD can lead teachers to adopt more engaging instructional 
practices. It offers readers access to all PD materials and invites them into dialogue about how to 


adapt the PD series and its implementation going forward. 
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Impacts of a Student Engagement-Focused 
High School Teacher Professional Development Series 

Improving postsecondary outcomes for historically underserved students requires 
substantial attention to creating the engaging classrooms that foster students’ motivation to learn 
(National Research Council, 2004; Fredricks et al., 2019). Secondary school classrooms are 
often characterized by such de-motivating factors as less student choice, little connection with 
the real world, poor student relationships with teachers, and more focus on whole-class 
instruction (Bridgeland et al., 2006; Cannata, 2013). As Fisher (2009) demonstrates from 
classroom observations, high school students spend most of their time just listening or waiting 
for instruction to happen, and not much time at all in the types of interactive learning experiences 
that are likely to be engaging. At best, most students simply tolerate high school by “posing, 
pretending, [and] waiting for the bell” (Pierce, 2006, p. 1); at-risk students respond by 
increasingly disengaging. 

While low levels of high school student engagement are a widespread national problem, 
they are especially concentrated in high poverty schools, which tend to have relatively fewer 
resources to address the higher levels of student need they encounter (e.g., Mirra & Rogers, 
2020). In such high poverty high schools “the curriculum and teaching often are unresponsive to 
the needs and interests of students” (National Research Council, 2004, p. 3). High rates of 
student failure are linked not only to students’ voluntary disengagement from school, but to the 
non-engaging quality of instruction they receive and the lack of caring relationships at school as 
well (Roderick et al., 2014). Students need classrooms characterized by “connective instruction 
practices” (Cooper, 2014, p. 366) that give them emotional connection to the teacher, the 


content, and the instruction in the class. Raising levels of student engagement requires 


“authentic academic work [that] involves students intellectually in a process of disciplined 
inquiry to solve meaningful problems, problems with relevance in the world beyond the 
classroom and of interest to them personally” (Marks, 2000, p. 158). As Woodworth and 
colleagues (2009, p. 36) point out, high poverty schools are much less likely than other schools 
to have teachers with “the ability to integrate real-world applications into lessons.” 

Despite a general consensus that improvement in student academic outcomes requires 
concentrated attention to teacher professional development (PD; Elmore, 2004; Bryk et al., 
2010), there has been limited previous research attention to equipping secondary teachers to 
engage students more effectively. A personalized coaching intervention focused on teacher— 
student interactions and student engagement has shown a positive effect on student behavioral 
engagement (Gregory et al., 2014) and teacher—student interactions and student achievement 
(Allen et al., 2015). Tumer and colleagues (2014) had some success with one group of teachers 
in a PD intervention designed to increase middle school students’ engagement in learning. An 
efficacy study of the intensive eMINTS PD intervention, which equips teachers to build a 
community of learners and engage students in inquiry-based learning, had evidence of a positive 
effect on student achievement (Meyers et al., 2016). At the same time, attempts to focus teacher 
attention on increasing student engagement have sometimes confronted teacher resistance to the 
possibility that instructional changes could influence student engagement (Bobis et al., 2016; 
Cooper et al., 2016). Similarly, the limited research on teacher PD related to grading practices 
has yielded mixed results (Olsen & Buchanan, 2019). Our review revealed the need for 
additional work in designing PD for high school teachers to help them to engage their students 


more effectively in learning. 


Designing PD to Help High School Teachers Engage Students More Effectively 

Our design process sought to create ways to focus teachers’ attention on what motivates 
students. We wanted to help teachers create classroom environments and learning experiences 
that would draw students into the learning process so that they will exert the effort required to be 
successful learners. Building on the foundation of considerable research about the factors 
associated with student motivation and engagement, we focused on the importance of fostering 
positive relationships between teachers and students as well as among students (e.g., Quin, 2017; 
Nguyen, Cannata, & Miller, 2018; Roorda, Koomen, Spilt, & Oort, 2011; Ruzek et al., 2016), 
providing choices and affirming student autonomy (Fredricks et al., 2019; Pendergast & Kaplan, 
2015), emphasizing student growth in mastery or competence (Moeller, Theiler, & Wu, 2012; 
Pendergast & Kaplan, 2015), and helping students to see the relevance of classroom learning for 
their current lives and future work (Hulleman & Harackiewicz, 2009; O’ Keefe, Horberg, & 
Plante, 2017). It is essential to help teachers to improve their instructional practices in ways that 
will create positive relationships and a vibrant learning community, give students more choices, 
help students to track their own growth in mastery, and relate classroom learning to students’ 
broader life experiences and future work lives. 

Given the teacher grading systems we had encountered over the years that tended to 
discourage students and convince them to stop trying (e.g., Farrington, 2014), we aimed to 
creatively address the issue of grading as well. We sought to provide teachers with opportunities 
to consider and discuss how different grading strategies affect student willingness to continue 
exerting effort to grow in their mastery and performance (Dueck, 2014). In contrast to those who 


view grading reforms as nonproductive manipulation of student outcomes, we built on research 


that demonstrates the relationship between student motivation and grading practices (e.g., 
Farrington, 2014). 

Our development process spanned two years in two different high schools. We tried out 
ideas with high school teachers during PD sessions, collected their feedback about what they 
found helpful and adjustments or changes they recommended, and revised sessions to test out 
again. Our diverse development team of research faculty, graduate students, and PD providers 
with extensive teaching experience engaged in lively debates as we discussed, beta-tested, and 
revised our sessions and modules repeatedly. 

The “Engaging High School Students in Academic Work” PD series we designed through 
this process sought to engage high school teachers across almost all subject areas in collaborative 
learning and reflection about the process of engaging students in the rigorous academic work 
required in high school. In particular, the PD series involved interactive teacher experiences and 
practical tools organized around these themes: 1) Building Relationships that Motivate Students; 
2) Motivating Students for Academic Work; 3) Assessment Strategies that Motivate Students (2 
sessions); 4) Building a Community of Learners: Strategies for Effective Cooperative Learning 
and Small Group Work; 5) Connecting the Work to Students’ Future Work Lives: Creating 
Assignments with Student Products for Real Audiences; 6) Reflecting on Assessment Strategies; 
7) Engaging Students in Critical Reading Skills; 8) Using Project-Based Learning to Deepen 
Learning and Engagement; and 9) Reflecting on Student Work, Student Progress, and Next Steps 
for Continuous Improvement. A customized but parallel series for math teachers focused 
particularly on engaging students in authentic problem solving and academic tenacity. The pilot- 


tested version of the PD series and associated tools are available for download (in modifiable 


form) on the following website: www.engage.everylgraduates.org. 


The design of the PD series sought to incorporate key components of effective PD 
identified in recent research, particularly a focus on active learning, delivered in a sustained 
format to a collective group of teachers from the same school grouped by subject areas (e.g., 
Desimone, 2009; Desimone & Garet, 2015; Opfer & Pedder, 2011). Sessions were designed to 
model the kinds of engaging classroom activities that teachers could take back to their own 
classrooms. The entire series was designed to give teachers 16 hours of PD over the course of an 
entire year, a dosage level found to be effective in meta-analysis findings (Yoon et al., 2007) and 
more possible to deliver in practice than longer interventions that could potentially be more 
effective but would need considerably more resources to sustain. 

Pilot Study Research Questions 

In the remainder of this article, we describe the pilot study we conducted to test the 

impact of the PD series. The study addressed the following research questions: 
e To what extent did treatment school teachers differ from control teachers on self-reported 
teaching practices after the intervention, controlling for baseline responses? 
e To what extent did treatment school students differ from control students in their 
assessment of teacher practices after the intervention, controlling for baseline responses? 
e To what extent did treatment school students differ from control students in their course 
passing rates after the intervention, controlling for baseline levels? 
After summarizing the results of that study, we offer some conclusions about conducting this 
kind of PD with teachers. We invite readers to try out the PD modules for themselves and 
engage in dialogue with us about how to make progress in engaging high school students more 


effectively going forward. 


Method 
Participants and Conditions of Design 

The district that agreed to have two of its high schools with similar demographic 
characteristics participate in the pilot study is located in a city of about 100,000 surrounded by 
several small towns and a large rural area in the U.S. southwest. The district serves just over 
10,000 students in grades 7-12. A large majority (85%) of its student population is economically 
disadvantaged. One in ten students (10.3%) is an English language learner. The district’s 
graduation rate was 78.4%. 

Because of particular district priorities regarding the two high schools willing to 
participate, it was not possible to randomize the schools to treatment and control conditions. The 
district made the assignment decision. Table 1 summarizes how the treatment and comparison 
schools compare demographically and on certain achievement and outcome measures. The 
demographic distribution of students at the two schools was similar. Although the treatment 
school had a higher graduation rate (86.7%) than the comparison school (79.2%), it had a lower 
percentage of students meeting standards on state tests than did the comparison school. This 
could be related to its higher percentage of special education students. 

Intervention Procedures 

The implementing high school agreed to dedicate a full day of PD time before the 
opening of the school year and to use many of its early release days for the PD series. This 
ensured that the sessions were held during time that faculty were expected to attend meetings and 
that the majority of the teaching faculty participated in all the sessions. Our research and 
development team delivered four one-hour sessions during the full-day PD in August, and 


delivered six additional 70-90 minute sessions approximately monthly throughout the school 


year. The total time teachers spent in PD was approximately 13 hours (as it was not possible to 
deliver the full 16 hours). 

Most teachers (except mathematics teachers) received ten general sessions. Because five 
of these sessions were delivered to smaller groups of teachers organized into departments, we 
were able to tailor several of the general sessions so that they fit the content orientation of these 
smaller groups (science content or English/history content). The PD series that math teachers 
received was even more customized: they participated in four of the whole-school general 
sessions and six math-focused breakout sessions. 

Data Sources 

Teachers at the two schools completed paper surveys at the beginning and end of the 
intervention year during the first and final PD sessions. The survey, administered by the PD 
delivery team (including the research team), had 56 items and took about 15 minutes to 
complete. Although response rates were relatively high for both Fall and Spring teacher surveys 
(above 75%), many treatment school teachers completed the Spring survey without providing a 
legible study “alias” so that the pre- and post-surveys could be matched. As a result, the 
percentage of teachers with reliably matched data for both surveys was lower (37% at treatment 
school, 63% at comparison school). Sensitivity analyses indicated that the “matchable” 
treatment group teacher respondents in the analysis sample may not be fully representative of all 
the teachers at the school. In particular, the group of treatment group teachers whose pre- and 
post-survey data could be matched were somewhat more likely than the full group of teachers 
taking the Spring survey to have more years of experience teaching. Treatment group math and 
science teachers were more likely than English and social studies teachers to be included in the 


analysis sample with both pre- and post-intervention survey data. 


Students at the two schools completed paper surveys at the beginning and end of the 
intervention year. The student survey had 74 items and took about 20 minutes to complete. 
Surveys were distributed to teachers by the study research assistant and administered during 
mathematics class by the classroom teacher. Response rates for students completing both 
surveys were 47% (n=856) at the treatment school and 46% (n=858) at the comparison school, 
reflecting issues with student mobility and attendance rates at these schools. 

The district provided administrative data including student demographic data, attendance 
and course grades. Because data from the year immediately before intervention were not 
available for ninth grade students at the treatment school (because they had attended a middle 
school in a different district), the analytical dataset included students in grades 10-12 for whom 
data were available in the baseline and intervention years (1103 students at the treatment school 
and 1125 students at the comparison school). This represented 76% of the total number of 
students in grades 10-12 enrolled during the intervention year at each school, and 90% of the 
total number of students enrolled for at least 160 days at each school in 2017-18." 

Measures and Analytic Methods 

Measures included both survey responses and student outcomes from administrative data. 
We constructed several scales from multiple survey items for teacher-reported instructional 
practices and student-reported perceptions of teacher practices. These included teacher-reported 
use of practices related to: 1) making connections to the real world; 2) giving students choices; 
3) making learning interesting; 4) helping students focus on learning goals and mastery; 5) 


giving students extra help; 6) building a classroom community; 7) building relationships with 


Tn order for the student course passing measures to be reliable, the analytic sample included only students who 
were enrolled for at least 160 of the 180 days of the school year. Ten percent of the students in the specified 
minimum enrollment group at each school had not been enrolled at the school the previous year and had no baseline 
data. 
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students; and 8) using motivating grading practices. Student-reported perceptions included 
views of how much teachers 1) care about them; 2) help them learn; and 3) use engaging 
teaching practices; and 4) how students describe their own academic effort. Details about these 
scales can be found in Mac Iver, Mac Iver, and Clark (2020). Using the course grades data, we 
constructed several outcome variables related to course passing. Dichotomous outcome 
measures included: passed all courses, failed no more than one course, passed math course, and 
passed English course. We created a “proportion of credits earned” variable by dividing the 
number of credits earned by the total number of credits attempted for the year. 

Analyses to estimate baseline equivalence used a measure of standardized mean 
difference between the groups at the beginning of the year, before the beginning of the PD 
series.* We used regression analysis® to estimate the relationship between treatment and 
outcome variables, controlling for prior year measures on student outcome variables, 
demographic measures (gender, ethnicity, special education status, and ELL status), and prior 
year’s attendance. 

Findings 

Overall, the results of the study provide only limited evidence of an impact of the PD on 
teacher practices. There was growth on only one measure of engaging teaching practices — 
giving students extra help — among teachers who participated in the PD, compared to teachers 
who did not. Students whose teachers participated in the PD series did have better course- 


passing outcomes than comparison students, which could have been related to extra help they 


2 Glass’s delta is calculated as the difference between treatment and comparison group means, divided by the 
standard deviation of the comparison group. 

3 For numeric dependent variables we used ordinary least squares regression. For dichotomous dependent variable 
measures, we used logistic regression and reported effect sizes as odds ratios. Although we use the common term 
“effect size,” we acknowledge that it is not appropriate to draw causal conclusions from this study that does not 
meet all the requirements for a rigorous quasi-experimental study. 
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received. At the same time, the study cannot provide causal evidence that students’ better course 
outcomes can be attributed to changes in instructional practices related to the PD intervention 
itself, especially given the lack of evidence that students perceived changes in instructional 
practices over the year. In the following section we provide more details about these findings. 
We then discuss factors that help to explain these findings. 

The study found large pre-existing differences in levels of teacher-reported practices, 
with treatment school teachers reporting higher frequency usage of most of the practices before 
the intervention began (Table 2). This meant we were only able to examine the intervention’s 
relationship to three practices: a focus on learning goals and mastery, giving extra help when 
needed, and grading practices. Regression analyses indicated that treatment school teachers 
reported “giving students extra help” substantially more often during the year than did teachers 
in the comparison group (see Table 3). In contrast, there was no difference between teachers in 
the treatment and comparison schools for “setting learning goals.” There was a nonsignificant 
difference in the use of “motivating grading practices.” 

In contrast to the differences in teacher-reported practices, analyses indicated that student 
perceptions about teacher practices before the intervention were nearly equivalent at the 
treatment and control schools (Table 4). For each of the student-reported teacher practices 
scales, we used regression analyses to investigate the relationship of the treatment condition with 
students’ spring perceptions, controlling for baseline perceptions. Contrary to our expectation, 
comparison school students reported significantly higher levels of teachers helping, teachers 
caring, and teachers using engaging classroom practices, though effect sizes were relatively 


small (Table 5). 
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In analyses of students’ pre-intervention outcomes, the two schools were not equivalent 
at baseline on attendance rate, with treatment school students having better prior attendance than 
comparison school students. But the standardized mean differences between the treatment and 
control schools on all five measures of prior course performance met baseline equivalence 
standards (Table 6). 

Table 7 summarizes the treatment’s “effects” on the five measures in our pre-specified 
primary outcome domain of course performance. In regression analyses that controlled for prior 
year’s outcome, prior year’s attendance, gender, ethnicity, special education status, and English 
language learner status, there were significant positive relationships between treatment and four 


39 66. 


of the five outcomes in this domain: “failed no more than one course,” “passed English course,” 
“proportion of credits earned,” and “passed all courses.”* Compared to students in the 
comparison school, students in the treatment school earned 1.5% more of the credits they 
attempted and were 53% more likely to pass their English courses, 45% more likely to fail no 
more than one course, and 26% more likely to pass all their courses. However, students at the 
treatment school were no more likely than students at the comparison school to pass a math 
course. 
Discussion 

The purpose of this study was to examine whether implementation of the designed PD 

series was associated with positive changes in teacher instructional practices and student 


academic outcomes. Despite its limitations, which we discuss in more detail below, this 


exploratory study yielded some promising results that should encourage use and additional study 


4 This was the case even when accounting for multiple comparisons by using Benjamini-Hochberg critical p-values. 
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of similar PD interventions. In particular, the study found a significant positive treatment effect 
on a key instructional practice and positive treatment impacts on student course performance. 

Treatment school teachers reported higher levels of growth on measures of “giving 
students extra help” over the course of the year than did comparison school teachers. The PD 
series attended by treatment teachers emphasized the importance of building good relationships 
with students and providing the opportunities needed for students to master their learning goals 
and recover from their less successful previous assessments. It is possible that this dimension of 
teaching was stressed by the treatment school administration and that the difference between the 
schools was not primarily attributable to the PD intervention. There was no similar treatment 
effect on other teaching practices emphasized during the PD series. Our reported treatment 
effect on teachers’ practices of giving students extra help is promising, but it is important to 
explore how implementation of the PD might be adapted and strengthened to result in more 
engaging instructional practices. 

The fact that treatment school students had better course passing outcomes than 
comparison students, controlling for their prior course performance, is an encouraging outcome, 
even though the student survey results did not uncover positive treatment effects on student 
perceptions of teacher practices. It is possible that the extra help provided by treatment school 
teachers contributed to the better course passing outcomes found in the study. At the same time, 
the fact that this study did not find evidence of improved student perceptions of instructional 
practices suggests the need for additional exploration of the process through which PD may help 
to improve instruction and students’ perceptions of it. 

Given the particular focus on math instruction in the additional content-specific PD 


sessions delivered as part of this study, it is important to explore the non-significant treatment 
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effect on math course passing (in contrast to the positive relationship of treatment to other course 
passing measures). We examined the fall to spring change in views on grading practices 
(propensity to provide more feedback and opportunities to redo assignments to demonstrate 
mastery) for math and English teachers. Although the number of teachers with both pre- and 
post-intervention survey data was somewhat limited, treatment math teachers had a larger 
negative growth on the grading practices measure than did comparison math teachers (—.27 vs. — 
.02; effect size? = —.41), while treatment English teachers had a smaller negative growth on that 
measure than did comparison English teachers (—.09 vs. —.17; effect size =.12). Over the course 
of the year, treatment school math teachers appeared to become more opposed to reform-oriented 
grading practices than did math teachers at the comparison school or other teachers at their own 
school. 

Even if the PD led math teachers’ instructional practices to became somewhat more 
engaging, this may not have sufficiently influenced their students’ course passing rates. This 
finding may indicate the importance of continued investigation of the impact of grading practices 
on student motivation and effort during the course of the semester. Helping students to maintain 
the belief that continued effort can result in growing mastery and increased success may be a 
particularly crucial instructional practice to address among high school teachers — one that 
requires more focused attention than our PD series was able to give. Expanding teachers’ 
opportunities to understand the impact of grading practices on student motivation may be an 
important next step to consider in teacher PD. 

Although PD can theoretically lead to changes in teacher beliefs and practices, we 


recognize that it does not always necessarily achieve its ultimate aim of improved student 


5 Glass’s delta used as effect size. 
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learning and outcomes. Desimone outlines a theory of change that shows how high-quality PD 
can “increase teacher knowledge and skills” and lead to a “change in attitudes and beliefs,” 
which in turn leads to a “change in instruction” and “improved student learning” (Desimone, 
2009, p. 185). But as Opfer and Pedder (2011) point out, teacher learning occurs within a 
complex, nested system of relationships, and causal arrows generally flow in more than one 
direction (as well as in cycles over time). Changes in teacher practice may actually precede 
changes in beliefs in some cases, as Guskey (2000, 2002) has argued and subsequent studies 
have shown (e.g., Hayes et al., 2019). PD may inspire teachers to make short-term changes to 
their practices before it changes their beliefs and attitudes. Even when teachers are engaged in 
PD that incorporates the key components for effectiveness, studies of the impact of this PD on 
student outcomes have yielded mixed results. 

Although there is evidence from randomized studies of the impact of middle school 
science PD on student knowledge gains (Penuel et al., 2011), of teacher study groups on early 
reading achievement (Gersten et al., 2010), and of one-to-one coaching of teachers on students’ 
behavioral engagement (Gregory et al., 2014), other large randomized studies of intensive PD 
efforts have yielded little or no evidence of improved teacher instructional practice or student 
achievement results (Garet et al., 2008; Garet et al., 2011). A study of several large districts 
found no relationship between teacher improvement in instructional practice (measured by 
teacher evaluation system scores) and teacher participation in PD (The New Teachers Project, 
2015). Many challenges remain as leaders seek to help teachers grow in their ability to engage 
students effectively in instruction that leads to growth in learning and improved student 


outcomes. 
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Limitations 

There are several limitations in this small pilot study. It involved only two high schools, 
which could not be randomly assigned to treatment or comparison conditions. Ninth graders 
could not be included in analyses of student outcomes because data from the prior year were not 
available for students from both schools. There were differences in teacher-reported practices 
and attitudes between the schools at baseline that did not permit some of the planned analyses. 
Although teacher survey response rates were greater than 75% at both administrations, teacher 
desires for anonymity prevented matching a large proportion of surveys at the treatment school 
and resulted in a much smaller effective response rate than desirable. There were probably 
unmeasured differences between the treatment and comparison schools in what teachers and 
students were experiencing during the intervention year that could have affected the results we 
found. All the reported results must be interpreted with caution until replicated more broadly. 

In addition, we believe that the delivery of the PD series by our team — a group of 
external providers from the other side of the country who flew in once a month but did not have 
ongoing relationships with the teachers in the high school — was a major limitation. The kind of 
ongoing change we were seeking to encourage almost certainly requires leadership within the 
school building that can follow up with teachers regularly, encouraging ongoing collaborative 
discussion and sharing of experiences among teachers who are seeking to try out new practices 
in the classroom. 

Conclusions and Recommendations 

This exploratory study contributes to a small but growing literature on PD interventions 

designed to increase adolescent student motivation and engagement in academic work. Given 


increased attention to the importance of student engagement in improving student learning 
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outcomes, it is essential to find ways to help teachers improve their instructional practice. We 
need to build more learning communities that address students’ needs for autonomy, belonging, 
and choice. Students need more engaging learning opportunities that help them to see the 
relevance of academic instruction to their lives and future work. Continuous improvement 
efforts focused on how to help teachers engage high school students more effectively in 
academic work are definitely needed. 

We invite readers to participate in such efforts by exploring, adapting, and using the PD 
resources we developed (available for download on www.engage.everylgraduates.org), and then 
joining into dialogue with us (see contact information below) about how the content or PD 
implementation strategies could be adapted in ways that could lead to positive teacher growth in 
instructional practices and increased student engagement in learning. We hope this may result in 
the formation of a networked learning community that can lead to significant improvements in 
the extent to which high school students are engaged in learning and successful in their academic 


work. 
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